Semi-Supervised Confidence-Level-based Contrastive Discrimination for Class-Imbalanced Semantic Segmentation

To overcome the data-hungry challenge, we have proposed a semi-supervised contrastive learning framework for the task of class-imbalanced semantic segmentation. First and foremost, to make the model operate in a semi-supervised manner, we proposed the confidence-level-based contrastive learning to achieve instance discrimination in an explicit manner, and make the low-confidence low-quality features align with the high-confidence counterparts. Moreover, to tackle the problem of class imbalance in crack segmentation and road components extraction, we proposed the data imbalance loss to replace the traditional cross entropy loss in pixel-level semantic segmentation. Finally, we have also proposed an effective multi-stage fusion network architecture to improve semantic segmentation performance. Extensive experiments on the real industrial crack segmentation and the road segmentation demonstrate the superior effectiveness of the proposed framework. Our proposed method can provide satisfactory segmentation results with even merely 3.5% labeled data.Comment: IEEE Cyber 2022 (Oral

FAC: 3D Representation Learning via Foreground Aware Feature Contrast

Contrastive learning has recently demonstrated great potential for unsupervised pre-training in 3D scene understanding tasks. However, most existing work randomly selects point features as anchors while building contrast, leading to a clear bias toward background points that often dominate in 3D scenes. Also, object awareness and foreground-to-background discrimination are neglected, making contrastive learning less effective. To tackle these issues, we propose a general foreground-aware feature contrast (FAC) framework to learn more effective point cloud representations in pre-training. FAC consists of two novel contrast designs to construct more effective and informative contrast pairs. The first is building positive pairs within the same foreground segment where points tend to have the same semantics. The second is that we prevent over-discrimination between 3D segments/objects and encourage foreground-to-background distinctions at the segment level with adaptive feature learning in a Siamese correspondence network, which adaptively learns feature correlations within and across point cloud views effectively. Visualization with point activation maps shows that our contrast pairs capture clear correspondences among foreground regions during pre-training. Quantitative experiments also show that FAC achieves superior knowledge transfer and data efficiency in various downstream 3D semantic segmentation and object detection tasks.Comment: 11 pages, IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (CVPR 2023), the work is mainly supported by the Natural Science Foundation Project of Fujian Province (2020J01826

SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation

In this paper, we propose a scribble-based video colorization network with temporal aggregation called SVCNet. It can colorize monochrome videos based on different user-given color scribbles. It addresses three common issues in the scribble-based video colorization area: colorization vividness, temporal consistency, and color bleeding. To improve the colorization quality and strengthen the temporal consistency, we adopt two sequential sub-networks in SVCNet for precise colorization and temporal smoothing, respectively. The first stage includes a pyramid feature encoder to incorporate color scribbles with a grayscale frame, and a semantic feature encoder to extract semantics. The second stage finetunes the output from the first stage by aggregating the information of neighboring colorized frames (as short-range connections) and the first colorized frame (as a long-range connection). To alleviate the color bleeding artifacts, we learn video colorization and segmentation simultaneously. Furthermore, we set the majority of operations on a fixed small image resolution and use a Super-resolution Module at the tail of SVCNet to recover original sizes. It allows the SVCNet to fit different image resolutions at the inference. Finally, we evaluate the proposed SVCNet on DAVIS and Videvo benchmarks. The experimental results demonstrate that SVCNet produces both higher-quality and more temporally consistent videos than other well-known video colorization approaches. The codes and models can be found at https://github.com/zhaoyuzhi/SVCNet.Comment: accepted by IEEE Transactions on Image Processing (TIP

Outram: One-shot Global Localization via Triangulated Scene Graph and Global Outlier Pruning

One-shot LiDAR localization refers to the ability to estimate the robot pose from one single point cloud, which yields significant advantages in initialization and relocalization processes. In the point cloud domain, the topic has been extensively studied as a global descriptor retrieval (i.e., loop closure detection) and pose refinement (i.e., point cloud registration) problem both in isolation or combined. However, few have explicitly considered the relationship between candidate retrieval and correspondence generation in pose estimation, leaving them brittle to substructure ambiguities. To this end, we propose a hierarchical one-shot localization algorithm called Outram that leverages substructures of 3D scene graphs for locally consistent correspondence searching and global substructure-wise outlier pruning. Such a hierarchical process couples the feature retrieval and the correspondence extraction to resolve the substructure ambiguities by conducting a local-to-global consistency refinement. We demonstrate the capability of Outram in a variety of scenarios in multiple large-scale outdoor datasets. Our implementation is open-sourced: https://github.com/Pamphlett/Outram.Comment: 8 pages, 5 figure

Integrative analyses of a mitophagy-related gene signature for predicting prognosis in patients with uveal melanoma

We aimed to create a mitophagy-related risk model via data mining of gene expression profiles to predict prognosis in uveal melanoma (UM) and develop a novel method for improving the prediction of clinical outcomes. Together with clinical information, RNA-seq and microarray data were gathered from the Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases. ConsensusClusterPlus was used to detect mitophagy-related subgroups. The genes involved with mitophagy, and the UM prognosis were discovered using univariate Cox regression analysis. In an outside population, a mitophagy risk sign was constructed and verified using least absolute shrinkage and selection operator (LASSO) regression. Data from both survival studies and receiver operating characteristic (ROC) curve analyses were used to evaluate model performance, a bootstrap method was used test the model. Functional enrichment and immune infiltration were examined. A risk model was developed using six mitophagy-related genes (ATG12, CSNK2B, MTERF3, TOMM5, TOMM40, and TOMM70), and patients with UM were divided into low- and high-risk subgroups. Patients in the high-risk group had a lower chance of living longer than those in the low-risk group (p < 0.001). The ROC test indicated the accuracy of the signature. Moreover, prognostic nomograms and calibration plots, which included mitophagy signals, were produced with high predictive performance, and the risk model was strongly associated with the control of immune infiltration. Furthermore, functional enrichment analysis demonstrated that several mitophagy subtypes may be implicated in cancer, mitochondrial metabolism, and immunological control signaling pathways. The mitophagy-related risk model we developed may be used to anticipate the clinical outcomes of UM and highlight the involvement of mitophagy-related genes as prospective therapeutic options in UM. Furthermore, our study emphasizes the essential role of mitophagy in UM

Masked Spatial-Spectral Autoencoders Are Excellent Hyperspectral Defenders

Deep learning methodology contributes a lot to the development of hyperspectral image (HSI) analysis community. However, it also makes HSI analysis systems vulnerable to adversarial attacks. To this end, we propose a masked spatial-spectral autoencoder (MSSA) in this paper under self-supervised learning theory, for enhancing the robustness of HSI analysis systems. First, a masked sequence attention learning module is conducted to promote the inherent robustness of HSI analysis systems along spectral channel. Then, we develop a graph convolutional network with learnable graph structure to establish global pixel-wise combinations.In this way, the attack effect would be dispersed by all the related pixels among each combination, and a better defense performance is achievable in spatial aspect.Finally, to improve the defense transferability and address the problem of limited labelled samples, MSSA employs spectra reconstruction as a pretext task and fits the datasets in a self-supervised manner.Comprehensive experiments over three benchmarks verify the effectiveness of MSSA in comparison with the state-of-the-art hyperspectral classification methods and representative adversarial defense strategies.Comment: 14 pages, 9 figure

Complete genome analysis of Bacillus velezensis TS5 and its potential as a probiotic strain in mice

IntroductionIn recent years, a large number of studies have shown that Bacillus velezensis has the potential as an animal feed additive, and its potential probiotic properties have been gradually explored.MethodsIn this study, Illumina NovaSeq PE150 and Oxford Nanopore ONT sequencing platforms were used to sequence the genome of Bacillus velezensis TS5, a fiber-degrading strain isolated from Tibetan sheep. To further investigate the potential of B. velezensis TS5 as a probiotic strain, in vivo experiments were conducted using 40 five-week-old male specific pathogen-free C57BL/6J mice. The mice were randomly divided into four groups: high fiber diet control group (H group), high fiber diet probiotics group (HT group), low fiber diet control group (L group), and low fiber diet probiotics group (LT group). The H and HT groups were fed high-fiber diet (30%), while the L and LT groups were fed low-fiber diet (5%). The total bacteria amount in the vegetative forms of B. velezensis TS5 per mouse in the HT and LT groups was 1 × 109 CFU per day, mice in the H and L groups were given the same volume of sterile physiological saline daily by gavage, and the experiment period lasted for 8 weeks.ResultsThe complete genome sequencing results of B. velezensis TS5 showed that it contained 3,929,788 nucleotides with a GC content of 46.50%. The strain encoded 3,873 genes that partially related to stress resistance, adhesion, and antioxidants, as well as the production of secondary metabolites, digestive enzymes, and other beneficial nutrients. The genes of this bacterium were mainly involved in carbohydrate metabolism, amino acid metabolism, vitamin and cofactor metabolism, biological process, and molecular function, as revealed by KEGG and GO databases. The results of mouse tests showed that B. velezensis TS5 could improve intestinal digestive enzyme activity, liver antioxidant capacity, small intestine morphology, and cecum microbiota structure in mice.ConclusionThese findings confirmed the probiotic effects of B. velezensis TS5 isolated from Tibetan sheep feces and provided the theoretical basis for the clinical application and development of new feed additives